Accessibility settings

This paper is in the following e-collection/theme issue:

User-Centered Design Case Studies (842) Web-based and Mobile Health Interventions (5805) Human Factors and Usability Case Studies (1292) mHealth for Wellness, Behavior Change and Prevention (4246) User Needs and Competencies (580) Internet Access and Digital Technology Use in an Elderly Population (599)

Published on in Vol 13 (2026)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/87365, first published .
Elderly woman with white hair using a smartphone, hand on chest.

Optimizing Usability of Digital Health Interventions for Nondigitally Native Adults: A Framework-Based Approach to Develop Just-in-Time Adaptive Interventions (JITAIs) and Other Adaptations

Optimizing Usability of Digital Health Interventions for Nondigitally Native Adults: A Framework-Based Approach to Develop Just-in-Time Adaptive Interventions (JITAIs) and Other Adaptations

Authors of this article:

Abby L Cheng1 Author Orcid Image ;   Christine Y Gou1 Author Orcid Image ;   Adriana Martin1 Author Orcid Image ;   Sarah M Hartz2 Author Orcid Image ;   Joanna Abraham3 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    1Division of Musculoskeletal Physical Medicine and Rehabilitation, Department of Orthopaedic Surgery, Washington University in St. Louis, Campus Box MSC 8233-0004-05, 660 S Euclid Ave, St. Louis, MO, United States

    2Department of Psychiatry, Washington University in St. Louis, St. Louis, MO, United States

    3Department of Anesthesiology and Institute for Informatics, Washington University in St. Louis, St. Louis, MO, United States

    Corresponding Author:

    Abby L Cheng, MD, MSc


    Background: Health-related technology use among nondigitally native adults is becoming widespread. Nevertheless, digital health interventions are typically not designed with the unique usability needs and preferences of this population in mind. Furthermore, most of these middle-aged and older adults are managing multiple medical conditions, which can also impact their preferences related to digital health interventions. A prime opportunity to address multimorbidity in nondigital natives using a digital health intervention is the intersection between mental health and chronic pain.

    Objective: The goal of this study was to use established frameworks and end-user feedback to identify actionable, usability-related features that can be incorporated into existing digital health interventions and that are preferred among nondigitally native adults. We aimed to identify general usability adaptations, as well as just-in-time adaptive interventions (JITAIs).

    Methods: We conducted a qualitative usability study of a convenience sample using a hybrid inductive-deductive content analysis to evaluate an existing mental health app (Wysa for Chronic Pain). Participants were 45 years or older; reported at least moderate symptoms of depression or anxiety (Patient Health Questionnaire -9 ≥10 or Generalized Anxiety Disorder -7 score ≥10); and endorsed having pain at least most days in the past 3 months. The Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies was used to identify potential usability-related adaptations for the target population. Development of usability-related JITAIs was guided by Nahum-Shani’s pragmatic framework for JITAI development and the Behavioral Integration Technology model.

    Results: Forty-two participants completed usability testing (mean age 57, SD 8 years; women n=32, 76%). Participants identified numerous opportunities to optimize their user experience, primarily by ensuring the app clearly describes how all its features are intended to be used and by minimizing navigation burden. Specifically, participants requested clear, step-by-step orientation and navigation instructions, rather than a brief onboarding experience that relies on user-led exploration and familiarity with conventional app symbols. Participants also recommended a customized experience based on their unique usage patterns. The most common JITAI opportunity identified was to strategically reduce user notifications in order to reduce the risk of notification fatigue and subsequent complete disengagement with the app.

    Conclusions: Identifying actionable opportunities to improve usability and prompt engagement with “the right tool at the right time for the right person” holds promise to improve the effectiveness of digital health interventions across the age span. The usability-related refinement opportunities identified in this study are generalizable to other digital health interventions that are relevant to older users who are likely to be managing multimorbidity, are not digital natives, and are more likely than younger users to have physical or cognitive challenges. The integrated framework-based process described in this article can also serve as a model for optimizing other existing digital health interventions.

    JMIR Hum Factors 2026;13:e87365

    doi:10.2196/87365

    Keywords

    mental health; depression; anxiety; chronic pain; mHealth; usability; user experience; satisfaction; digital health; digital mental health intervention; just-in-time adaptive intervention; middle-age; older adult 


    Most digital health interventions are not equally “accessible” to everyone [1]. Specifically, they are typically not designed with the usability needs and preferences of nondigital natives (born before 1980) in mind [2,3]. Technology use is rapidly growing across the age span [4-6], but compared to younger users who grew up “digitally native” with widespread access to technology and for whom many interventions were initially designed, middle-aged and older adults tend to have different habits and characteristics. For instance, they tend to check their mobile devices less frequently, and most do not instinctively search for a technological solution to a problem. In addition, this older population often experiences age-related changes in visual acuity and eye tracking patterns, and some have age-related dexterity or cognitive deficits [4,7-10]. Furthermore, the prevalence of multimorbidity increases with age [11]. As a result, nondigitally native adults tend to prefer different screen layouts than younger technology users; they may benefit from more “push” notifications, and they are more often interested in interventions that address multimorbidity [4,5].

    A growing body of evidence is available to guide developers in understanding the usability preferences of nondigitally native adults [4,5,12], but a gap still exists in translating these general preferences into real interface changes that are implemented within existing digital health interventions. Furthermore, just-in-time adaptive interventions (JITAIs) are a relatively new and persistently underused method that aim to deliver the “right intervention at the right time to the right person,” and they have the potential to reduce the need for users to initiate interaction with a digital health intervention or to navigate through the intervention independently [13]. We hypothesize that JITAIs may be particularly impactful to improve usability for nondigitally native adults, but they are not yet widely applied to publicly available digital health interventions [14]. Guiding frameworks have also been developed in recent years that can facilitate optimal reach and effectiveness of digital health interventions, but they, too, have not been routinely used in intervention development [15]. Applying these frameworks and JITAI capabilities to adapt existing interventions for key target populations could rapidly improve the accessibility and subsequent effectiveness of publicly available digital health interventions.

    A prime opportunity to address multimorbidity in nondigitally native adults using a digital health intervention is at the intersection between mental health and chronic pain [16-21]. Digital mental health interventions are now widely available to address depression and anxiety symptoms [22-27], in large part because they can be effective and improve access to mental health support for people who otherwise may be limited by barriers such as stigma, cost, or geography [27-32]. Nevertheless, chronic pain commonly coexists with symptoms of depression and anxiety, and it reduces the effectiveness of mental health treatments unless pain is simultaneously addressed [20,33-38]. Furthermore, the prevalence of chronic pain increases with age [20,35].

    The goal of this study was to use established frameworks and end-user feedback to identify usability preferences among nondigitally native adults for digital health interventions. We specifically focused on nondigitally native adults with chronic pain and coexisting symptoms of depression or anxiety, and we aimed to obtain feedback on participants’ preferred general usability adaptations, as well as usability-related JITAIs. Our motivating scientific premise is that incorporating usability-related adaptations will improve overall engagement and subsequent effectiveness of digital health interventions, and evaluation of this potential relationship will be assessed in future work.


    Study Design

    This was a formative, qualitative usability study in which participants from the target population of interest completed usability testing of an existing mental health app. Usability testing occurred in 2024, and data analysis occurred in 2024 and 2025. Of note, additional feedback from study participants regarding contextual determinants of engagement with mental health apps was reported separately [12].

    This study was performed via an academic-industry partnership. Academic investigators had full independence in designing the study, recruiting participants, conducting the usability testing, performing data analysis, and reporting study results. The industry partner provided access to the mental health app under investigation and created mock-ups of refined app screens and workflows for the iterative usability testing process. The academic investigators shared the study findings with the industry partner so that the industry partner could refine its mental health app accordingly.

    Ethical Considerations

    Ethical approval was obtained by the Washington University Institutional Review Board via a waiver of written informed consent prior to participant recruitment (Institutional Review Board ID 202311024). Participant data were maintained in secure databases and were deidentified before sharing outside the institutional review board–approved study team. Participants received a US $30 gift card for participation.

    Participants or Users

    Participants were at least 45 years old, reported at least moderate depression or anxiety symptoms (Patient Health Questionnaire-9 ≥ 10, Generalized Anxiety Disorder-7 score ≥ 10, or both), and endorsed chronic pain (defined as pain on most days or every day in the past three months) [20,39-41].

    Exclusion criteria included endorsement of an active mental health crisis that warranted escalation of care (eg, active suicidal ideation and psychosis), cognitive impairment that would interfere with communication with the researcher or meaningful engagement with the mental health app (assessed informally by the research coordinator asking the potential participant to restate the study purpose, activities, and risks, benefits, and alternatives of study participation during the informed consent process), and residing outside the United States.

    Recruitment

    Participants were a self-selected convenience sample recruited from multiple sources including a chronic musculoskeletal pain clinic at a US academic medical center, email, and social media advertisement via the medical center’s volunteer research participant registry, and targeted advertisement in a free online health community (The Mighty). Eligibility screening was performed using a self-reported questionnaire hosted by REDCap (Research Electronic Database Capture; Vanderbilt University), followed by verification by the research coordinator [42,43]. Recruitment was closed when usability testing by additional participants no longer yielded new themes within the convenience sample during the iterative analysis process [44].

    Intervention Assessed in Usability Testing

    The mental health app assessed in usability testing, Wysa for Chronic Pain, aims to address mental health and chronic pain through a multifaceted approach by facilitating behavioral activation, pain acceptance, and improved sleep quality [45-48]. It guides users through evidence-based cognitive behavioral therapy, mindfulness-based interventions, and sleep support. The app has used a rule-based conversational agent with natural language understanding to deliver therapeutic content since 2016. In 2024, it augmented this approach with the addition of large language models at specific nodal points. Large language model–based capabilities were integrated into check-in conversations to improve the chatbot’s ability to understand context-specific content over multiple interactions and respond in a more personalized manner to the user. However, risk-based and crisis conversations have remained completely rule-based and written by a clinician. Although the parent Wysa company offers multiple commercially available products, Wysa for Chronic Pain is not currently commercially available.

    The therapeutic tools within Wysa for Chronic Pain are designed to be delivered via a combination of a structured curriculum and optional user-led exploration. By default, the app sends push notifications to users twice daily. The morning notification prompts an activity scheduling and planning task, and the evening notification prompts an activity tracking and mood reflection task and then offers a sleep meditation (Figure 1). Additional therapeutic content such as cognitive restructuring, thought recording, journaling, anxiety management, etc is delivered in a weekly cadence over 8 weeks. Each week, users receive a push notification that directs them to a report that details their app engagement thus far. Accessing the report also launches a congratulatory message for their engagement and unlocks access to new therapeutic tools as a reward for their engagement. Users can also independently navigate to therapeutic tools at any time, which are organized into “tool packs.” As a safety feature, the app also includes an “SOS” safety plan feature, which lists crisis hotlines and allows users to enter and store self-reported sources of hope and “safe people or places” to reference in moments of crisis.

    ‎
    Figure 1. Screenshots of Wysa for Chronic Pain.

    Usability Testing Process

    Participants answered sociodemographic questions via an electronic REDCap questionnaire for descriptive purposes. Usability tests were conducted and recorded using audio and visual conferencing via a secure Zoom (Zoom Video Communications, Inc.) platform, each lasting approximately 45‐60 minutes. After participants downloaded Wysa for Chronic Pain to their mobile device, they were asked to complete the app’s onboarding process and complete several tasks within the app, all while performing “think aloud” testing, during which they described their thoughts and experiences in real-time (Multimedia Appendix 1: interview guide). During this process, the facilitator also took note of participants’ hesitations and inaccuracies in navigating the app, and the facilitator requested feedback on potential JITAIs that could be incorporated into the app. JITAI options and their details were iteratively refined throughout the usability testing process via a rapid iterative testing process. Based on feedback from the first 21 participants, mock-ups of revised app screens and workflows were created and shared with the next 13 participants using Figma (Dev Mode) and Twine (Chris Klimas) software. Additional minor refinements were made to the mock-ups for review by the final 8 participants. However, the evaluation tasks and the app’s core functionalities remained consistent for all participants.

    Analysis Frameworks

    The Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) was used to identify and report potential usability-related adaptations needed for our target user population (Table 1) (Figure 2) [49]. FRAME-IS builds from the Framework for Reporting Adaptations and Modifications-Enhanced and is organized into modules to systematically document modifications to implementation strategies [50]. We elected to use the FRAME-IS rather than the Framework for Reporting Adaptations and Modifications-Enhanced because our goal was to adapt the delivery of therapeutic content within the digital health intervention (ie, usability and engagement optimizations as an implementation strategy), rather than adapt the core therapeutic content itself (ie, cognitive behavioral therapy, mindfulness exercises, etc as the evidence-based intervention).

    Table 1. Application of the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies, which guided proposed adaptations to improve usability of a mental health app for nondigitally native adults who have chronic pain.a
    FRAME-ISb module or sub-componentApplication of FRAME-IS to target intervention and population
    Module 1
    The intervention being implemented is:
    • Therapeutic tools within the Wysa for Chronic Pain digital platform
    The implementation strategy being modified is:
    • Improving usability of the intervention’s platform
    The modifications being made are:
    • General adaptations to the intervention’s interface
    • Incorporation of JITAIsc
    The reasons for the modifications are:
    • To lower the user’s navigation burden and frustration through the intervention’s interface
    • To increase the likelihood that the user is aware of, and engages with, relevant therapeutic tools
    Module 2
    What is modified?
    • Format-related contextual modifications (See Tables 2 and 3)
    Module 3
    What is the nature of the content, evaluation, or training modification?
    • Refining packaging of intervention content
    Optional: what is the relationship to core elements?
    • Fidelity-consistent: core elements of the intervention are preserved
    Module 4
    What are the goals?
    • Increase the usability and acceptability of the intervention to the target user group
    • Improve intervention fidelity and engagement such that users engage with therapeutic tools as intended (ie, improve retention)
    • Decrease disparities in intervention delivery (ie, improve uptake and retention by nondigitally native adults)
    What is the level of the rationale for the modifications?
    • Recipient (end-user)
    Module 5
    When is the modification initiated?
    • Scale-up phase (ie, after the intervention was developed but before it is made available to the general public)
    Is the modification planned?
    • Planned, proactive
    Module 6
    Who participates in the decision to modify?
    • Recipients (end-users), researchers, intervention developers
    Optional: who makes the ultimate decision:
    • Intervention developers (ie, based on feasibility and alignment with other intervention features)
    Module 7
    How widespread is the modification?
    • All recipients (end-users) of the intervention

    aThe left column outlines the elements of the FRAME-IS, and the right column outlines the application of the framework to this study’s target intervention and population..

    bFRAME-IS: Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies

    cJITAIs: just-in-time adaptive interventions

    Table 2. General usability adaptations proposed to improve usability of a mental health app for nondigitally native adults who have chronic pain.
    Name of adaptationDescription
    Onboarding conversationInclude a thorough, text-based description of how users are intended to engage with the app (eg, timing, frequency, content, goals of engagement). Also explain terms used by the app (eg, “roadmap,” “journal,” etc).
    Onboarding tourInclude a dynamic visual tour of how to navigate the app’s interface, including audio explanation and text captions.
    Therapeutic approachExplain the app’s therapeutic approach during the onboarding experience. For example, explain “how an app can help with pain” by differentiating pain interference from pain severity and by stating the goals of reducing pain interference and improving pain acceptance.
    Search barEnable users to rapidly search the library of therapeutic tools based on a tool’s name, category, or other feature.
    Clarify “SOS”Appropriately label an app’s crisis tools so users do not assume that pushing a button labeled “SOS” will automatically dial 911 (but rather, will direct to creation and reference of user-reported sources of hope and “safe people/places,” list of crisis hotlines, etc)
    Personalized “flare-up kit”Create easy reference to a personalized plan for symptom flares (such as what would be developed by the user during a standard cognitive behavioral therapy session). For example, pin a “preparing for pain flares” plan near existing tool packs, which lists users’ preferred nonpharmacologic pain management strategies (eg, ice and heat modalities, distraction techniques, mental relaxation techniques, etc).
    Pinned toolkitMaintain a group of in-app therapeutic tools that are useful for acute needs (eg, pain flares) and are always easily accessible from the app home screen (eg, mindfulness exercises, gentle stretches, reminder of user-reported preferred pain management strategies in the “flare-up kit”).
    Experience customizationCustomize and recommend therapeutic tools based on users’ responses during onboarding and subsequent conversations within the app (eg, facilitated by generative artificial intelligence).
    Other challenges to addressWhen prompting users to report challenges they are living with, include options for loneliness, health concerns, and finances (which impact overall well-being and should then be incorporated into customizing the user’s app experience).
    Response to tools within appTrack users’ self-reported response to each therapeutic tool, at least after first use. Use this data for future experience customization.
    Weekly symptom assessmentAssess symptoms weekly with a brief measure (eg, PHQ-4)a [51], to facilitate (app- and user-led) pattern recognition between symptoms and other events (eg, to facilitate future app experience customization and for users to reference while journaling).
    Weekly reportRelate symptom changes (eg, mood and pain) to app engagement (eg, which tools, how often), and allow users to add other external events to the report which they believe may have impacted their symptoms (eg, work, family events).
    Preparing for hard daysBefore closing the “weekly report” experience, ask users, “Do you expect any extra challenges this week?” If so, inquire about the timing and type of challenge. Offer to schedule a personalized tool in preparation for the challenge (eg, grounding exercise before a person with agoraphobia leaves home, meditation exercise for an anticipated activity-related pain flare, etc).
    Tip of the day or weekAs part of the daily check-in or weekly report, highlight a feature or tool that has not yet been explored by the user or that was rated highly but has not been used recently.
    Curriculum changeAfter a user completes the app’s structured curriculum, ask if the user prefers to continue structured check-ins versus transition to only user-led engagement with tools.

    aPHQ-4: Patient Health Questionnaire-4.

    Table 3. Usability-related JITAIs proposed to improve usability of a mental health app for nondigitally native adults who have chronic pain.
    JITAIa namePurposeTailoring variablesIntervention optionsProximal outcome(s)
    OnboardingIncrease onboarding completion ratesIf user downloaded app but did not complete onboarding within intended periodThen, send a notification that encourages the user to finish onboarding.
    • Completion of onboarding
    Short disengagedMotivate a user to engageIf no app engagement for short period (eg, 24 h)Then, send a notification that directly opens to a relevant tool.
    • Any engagementb
    • Content engagementc
    Long disengagedRespect that users’ preferred or needed frequency of engagement may fluctuate based on symptom severity and life eventsIf no app engagement for a prolonged period (eg, 7 d)Then, send an “FYI/optd back in”e notification, and stop all notifications for a brief period (eg, 3 d). After the brief period, send a notification that directly opens to a relevant tool.
    • Any engagementb
    • Content engagementc
    AM check-ins onlyPersonalize timing of content delivery to users’ demonstrated preferencesIf no engagement with PM check-ins but continued engagement with AM check-insThen, send an “FYI/opt back in”e notification, stop daily “PM check-in” notifications, and move the “activity tracking and mood reflection” task to the AM check-ins.
    • Engagement with the “activity tracking and mood reflection” task compared to a preceding time interval
    • Change in content engagementc compared to a preceding time interval
    PM check-ins onlyPersonalize timing of content delivery to users’ demonstrated preferencesIf no engagement with AM check-ins but continued engagement with PM check-insThen, send an “FYI/opt back in"e notification, stop daily “AM check-in” notifications, and move the “activity scheduling and planning” task to the PM check-ins.
    • Engagement with the “activity scheduling and planning” task compared to a preceding time interval
    • Change in content engagementc compared to a preceding time interval
    Sporadic engagement onlyPersonalize timing of content delivery to users’ demonstrated preferencesIf no engagement with AM or PM check-ins but continued user-led engagement at other timesThen, send an “FYI/opt back in”e notification, and stop daily “AM check-in” and “PM check-in” notifications. Instead, each day, send a single personalized notification that directly opens to a relevant tool.
    • Change in content engagementc compared to a preceding time interval
    Symptom health flarefOffer increased support when increased need is demonstratedIf symptom score (eg, PHQ-4g [51]) worsens compared to the preceding score and the user’s current push notification frequency is less than maximal frequencyThen, send an “FYI/opt out”e notification, and increase notification frequency back to maximal frequency (eg, twice daily).
    • Change in content engagementc compared to a preceding time interval
    AccessibilityReduce communication barriersIf user’s typo frequency interferes with effective communication with the app or if user’s typing speed is slower than a designated thresholdThen, send a notification that describes accessibility options (eg, increase font size, speech-to-text, text-to-speech) and offer to direct the user to their mobile device’s accessibility settings
    • Change in content engagementc compared to a preceding time interval

    aJITAI: just-in-time adaptive interventions

    b“Any engagement” is operationalized as opening the app during the specified target engagement time interval. This is a binary outcome.

    c“Content engagement” is operationalized as the total number of tools, conversations, and check-ins the user engaged with during the specified target engagement time interval. This is a continuous outcome.

    dFYI: for your information.

    eIf the “opt back in” or “opt out” option is selected, the rest of the intervention is not delivered.

    fThe “symptom flare” JITAI over-rides any engagement-related JITAIs which would otherwise reduce the frequency of push notifications to the user.

    gPHQ-4: Patient Health Questionnaire-4.

    ‎
    Figure 2. Integration of established frameworks to guide identification and development of usability-related adaptations. FRAME-IS: Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies; JITAI: just-in-time adaptive intervention; BIT: Behavioral Integration Technology

    Specifically for developing usability-related JITAIs, we applied Nahum-Shani’s pragmatic framework for JITAI development and the Behavioral Integration Technology (BIT) model [52,53]. Nahum-Shani’s pragmatic framework facilitated the systematic identification of opportunities for intervention using a JITAI [52]. That is, the framework was used to operationalize intended temporal progression of events through the app, which led to identification of promising opportunities for intervention with a JITAI. The BIT model facilitates effective translation of behavioral interventions into digital delivery systems. Specifically, we used the BIT model to design the JITAIs’ instantiation elements with intentional incorporation of behavior change strategies, implementation science principles, and user interface design principles that appeal to our target population [54].

    Analysis

    Using a framework-informed hybrid inductive-deductive content analysis approach [55], iterative versions of general usability adaptations and usability-related JITAIs were developed and refined throughout the usability testing process based on feedback from participants and mapping to the analysis frameworks. As usability tests were completed, audio recordings of the usability tests were professionally transcribed (Rev.com), cleaned, and chunked (CYG). Three members of the study team (ALC, CYG, AM) collaboratively conducted open coding of the first 5 transcripts using an inductive, data-driven approach to identify concepts emerging from the data. Through discussion and consensus, they developed an initial codebook, which was then applied independently to pilot code the next 3 transcripts. Interrater reliability for these three transcripts was approximately 90%, with fewer than 5 coding discrepancies across coders. Discrepancies were resolved through consensus, and the codebook was refined accordingly. As usability tests were conducted with additional participants, the finalized codebook was applied using an independent open coding approach to these additional transcripts, each of which was coded by a single team member (CYG or AM), with periodic cross-checks between coders to ensure coding consistency. Coding was managed using NVivo 15 (Lumivero). Three team members (ALC, CYG, and AM) collaboratively and iteratively grouped related codes into preliminary themes that reflected participants’ usability needs and suggested app adaptations, including JITAIs. This process was continued until analysis of additional transcripts no longer yielded new themes. Then, we conducted a deductive analysis by examining the inductively derived themes in relation to the FRAME-IS framework, Nahum-Shani’s pragmatic framework for JITAI development, and the BIT model. The 3 team members (ALC, CYG, and AM) reviewed and discussed the themes to determine their alignment with these guiding frameworks, refine interpretation where appropriate, and translate findings into a final set of proposed general usability adaptations and usability-related JITAIs.

    Reflexivity Statement

    Usability testing was facilitated by a senior female research coordinator who has clinical research experience working with older adults and in the fields of mental health and orthopedics (AM). Participants’ only contact with the facilitator was related to study recruitment, consent, and participation. The other 2 research team members who contributed to codebook development included a female physician scientist with a clinical background in physical medicine and rehabilitation (ALC), as well as a female physician trainee in physical medicine and rehabilitation residency (CYG). The clinical and musculoskeletal background of these team members may have influenced data collection and interpretation such that physical barriers and related adaptations were disproportionately considered over solutions that considered mental health and systems-level factors. We worked to balance this potential bias by incorporating input from a clinician scientist with mental health clinical expertise (SH) and a health information technology expert (JA).

    Rigor and Reproducibility

    Informal member checking was performed by the facilitator (AM), who paraphrased and restated participants’ feedback during the usability testing process to confirm that participants’ intent and insights were accurately understood. Additionally, preliminary themes that emerged from early interviews were shared with later participants to assess for perceived validity. Participants generally affirmed the face validity of these themes.


    Of the 62 people who completed the screening questionnaire and were determined to be eligible, 42 users participated in usability testing (mean age 57, SD 8 years; women: n=32, 76%; mean chronic pain duration 16, SD 12 years) (Table 4). Reasons for nonparticipation included (1) not responding to invitations to schedule an appointment (or missing a scheduled appointment) (n=4), and (2) achievement of thematic saturation prior to their participation (n=16).

    Table 4. Characteristics of usability testing participants (N=42).
    CharacteristicsValues
    Age (years), median (IQR; range)55 (51-61; 45-76)
    Sexa, n (%)
    Female32 (76)
    Male10 (24)
    Race, n (%)
    White30 (71)
    Black or African American9 (21)
    Other1 (2)
    More than one race1 (2)
    Unknown or prefer not to answer1 (2)
    Ethnicity, n (%)
    Hispanic or Latino3 (7)
    Not Hispanic or Latino36 (86)
    Unknown or prefer not to answer3 (7)
    Employment status, n (%)
    Working full-time11 (26)
    Working part-time4 (10)
    Disabled13 (31)
    Retired10 (24)
    Unemployed2 (5)
    Prefer not to answer2 (5)
    Frequency of internet useb, n (%)
    Most of the day12 (29)
    Multiple times a day28 (67)
    At least once a day1 (2)
    Less than once a day1 (2)
    High-impact chronic pain presentc, n (%)33 (80)
    BPI Pain Interference scored, median (IQR; range)7.1 (6‐8.1; 2.4‐9)
    PHQ-9e score (years), median (IQR; range)14 (12-17; 7-22)
    GAD-7f score (years), median (IQR; range)12 (9-15; 1-21)

    aSex and gender identity were concordant for all participants.

    bSelf-reported internet use was captured by the Pew Digital Savviness Classifier [56].

    c”High impact chronic pain” is defined as pain that limits life or work activities on most or every day over the past 3 months [20].

    dThe Brief Pain Inventory (BPI) Pain Interference score ranges from 0 to 10 [57]. Higher scores indicate greater pain interference.

    eThe Patient Health Questionnaire (PHQ)-9 is a 9-item screening measure for depressive symptoms [39]. Scores range from 0 to 27. Higher scores suggest more depressive symptoms.

    fThe Generalized Anxiety Disorder (GAD)-7 is a 7-item screening measure for anxiety symptoms [40]. Scores range from 0 to 21. Higher scores suggest more anxiety symptoms.

    We identified general usability adaptations, as well as usability-related JITAIs, that users perceived would enhance their app experience. Rapid, nearly universal convergence was achieved related to suggestions for general adaptations to provide a more thorough onboarding experience to the app (Table 2). Specifically, users requested clear, step-by-step navigation instructions, rather than a brief onboarding experience that relied on user-led exploration of the app and familiarity with conventional app symbols (eg, three vertical ellipsis dots indicating “more options”). A 49-year-old female stated:

    For those who may not be as familiar – and you’re going to start getting a lot of those much older users, especially in this kind of a world with health concerns – I don’t know that they would know what that [icon] means without a tour of the app first.

    Users also appreciated a customized app experience based on their unique challenges and app usage patterns, although specific desired customizations were somewhat varied, and no clear patterns emerged related to demographic characteristics or recruitment source, based on impressionistic observation.

    Consistent with Nahum-Shani’s pragmatic framework [52], the majority of participants favored that some of the usability-related JITAI opportunities be designed to strategically reduce notifications to the user in order to reduce the likelihood of a negative impact such as notification fatigue and subsequent complete disengagement with the app (Table 5). As one 60-year-old female participant said:

    If I’ve not engaged in a week and after three days of nothing bothering me – no notifications – I think that a notification out of the blue would probably kick me back into gear. Give me that little kick start I need.

    When a notification is delivered, application of the BIT model [53] facilitated identification of the notification’s instantiation components that were most appealing to users (Table 6).

    Table 5. Application of Nahum-Shani’s pragmatic framework which guided development of usability-related JITAIs to improve usability of a mental health app for nondigitally native adults who have chronic pain.
    Component of Nahum-Shani’s pragmatic frameworkApplication to target intervention and population
    Defining the problem
    Whom are you trying to help?
    • Nondigitally native adults, especially those who have chronic pain that coexists with symptoms of depression, anxiety, or both. This target population may be less likely than younger adults to readily and proficiently navigate through an app independently and find tools of interest.
    What is the desired distal outcome of the JITAI(cs)?
    • Improve depression and anxiety symptoms (primary outcomes)
    • Improve behavioral activation, pain acceptance, and sleep quality (proposed mediators of primary outcomes)
    What is the temporal progression of key factors towards the distal outcome?b
    • Morning engagement: for behavioral activation (eg, activity scheduling and planning)
    • Evening engagement: for behavioral activation (eg, activity tracking, mood reflection), sleep optimization (eg, meditation)
    • Weekly engagement: for reflection and introduction of new cognitive behavioral skills (eg, cognitive reframing, thought recording)
    • Sustained engagement (8 wk): for growth, maintenance, and eventual independence with cognitive behavioral and pain self-management skills
    What are contender proximal outcomes?
    • Tool-specific engagement within the app
    Defining just-in-time in the context of the identified problem
    What factors mark a state of vulnerability or opportunity?
    • Lack of engagement with the app (as intended or as previously engaged)
    What possible intervention options can affect the proximal outcomes?
    • A notification or nudge that is noticed by, and appeals to, the user
    What factors mark a state of unreceptivity to the selected intervention options?
    • Already engaging with the app as intended or more frequently
    • Not responding to existing notifications or nudges
    Formulating the adaptation strategy:
    What are the tailoring variables?
    • User’s current pattern of engagement
    For each possible level of the tailoring variable, which intervention option is likely to have the desired effect on the proximal outcome?
    • See Table 3. Additional research is needed to identify optimal cut-points for each tailoring variable.
    What plausible decision rules can be generated to operationalize effective adaptation?
    • See Table 3. Additional research is needed to identify optimal cut-points for each tailoring variable.

    aJITAI: just-in-time adaptive interventions.

    bThe temporal progression described reflects how users are intended to engage with the app.

    Table 6. Application of the BITa model which guided development of usability-related JITAIsb to improve usability of a mental health app for nondigitally native adults who have chronic pain.
    BIT componentApplication to target intervention and population
    Theoretical
    Why: aims
    Clinical aimsImprove depression and anxiety symptoms
    Usage aimsIncrease engagement with evidence-based therapeutic tools
    How (conceptual): behavior change strategies
    Clinical strategiesImprove behavioral activation (eg, through goal setting), pain acceptance (eg, through pain neuroscience education and mindfulness exercises), and sleep quality (eg, through sleep hygiene education and meditation exercises)
    Usage strategiesDirect user through a structured educational curriculum and on-demand access to therapeutic tools
    Instantiation
    What: elementsNotifications that are delivered with personalized timing and content based on passively and actively collected data
    How (technical): characteristicsEach push notification will feature a visual and audio alert, as permitted by the user’s self-selected mobile device settings. The notifications direct the user to relevant therapeutic tools, as identified from the user’s prior responses (eg, challenges expressed) and app engagement patterns. The content of the notifications is encouraging but not overly cheerful, with judicious use of emojis.
    When: workflowTiming of delivery of the push notifications is determined by event-based rules. However, precise timing relative to occurrence of the event(s) may be adaptive based on the user’s engagement history and other characteristics.

    aBIT: behavioral intervention technology

    bJITAI: just-in-time adaptive intervention.

    We identified 8 potential usability-related JITAIs to consider for build-out and formal effectiveness testing (Table 3). These JITAIs were developed to address the competing priorities of (1) providing predictable support to the user, and (2) acknowledging that pain and mood symptoms (and hence, a user’s need for support) can fluctuate. Users reported varying preferences regarding the frequency of push notifications and indications to change the push notification frequency. Therefore, our JITAIs that propose to change the frequency of future push notifications all include an “opt back in” or “opt out” option so that users maintain autonomy over their app experience. As one 48-year-old male described:

    Some people, if they’re in a certain mental state, may need a little bit more persistence, whereas other people may need more space. I feel like me personally – if I didn’t engage for a while and then had a [several-day break from notifications], it would be like, ‘Hey, we’re still here if you need us.’ I think that would work for me personally, but I think it would all depend on the individual. If the app gave you an option or the ability to change the [notification frequency] or suggest how you want to be taken care of, yeah, I think that would be a good idea.

    The precise text of each JITAI notification is intended to be engaging and varied over time, even if the same JITAI is triggered more than once. As relevant and possible, the JITAI notifications should leverage information that was previously reported by the user in order to personalize the content and resonate with the user (eg, suggesting tools based on prior challenges reported, tools that were previously rated as helpful, and tools that have not been used yet but are expected to be helpful based on the user’s reported challenges and prior tools used). When selected, most of the JITAI notifications are intended to directly open the app to a relevant tool for the user.


    Principal Results

    In this study, we identified adaptations to an existing mental health app that better meet the stated usability preferences and needs of nondigitally native adults who have chronic pain and coexisting symptoms of depression or anxiety. Study participants specifically identified several opportunities to optimize their user experience by ensuring that the digital health intervention clearly describes how all its features are intended to be used and by minimizing navigation burden. For instance, users proposed general usability adaptations for a more robust orientation experience during app onboarding and for ongoing personalized engagement prompts using usability-related JITAIs.

    Comparison With Prior Work

    Based on prior literature and feedback from participants in this study [15,58], when identifying decision rules for our proposed JITAIs, we aimed to balance the use of passively collected app engagement data and user-directed triggers in order to respect users’ dual desires for reduced navigation burden and also maintenance of autonomy and privacy. In related work [12], the majority of users in this target population expressed hesitation about allowing a digital mental health app to collect and use other data, such as physical location that is passively collected from their mobile device or other wearable sensors. Furthermore, older adults and those with multimorbidity are less likely to continuously carry a mobile device or use a wearable sensor, which would limit the impact of JITAIs that rely on datapoints from these sources [59-62]. As a result, the JITAIs developed in this study were intentionally designed to operate using only engagement-related and user-reported data that is collected within the app itself.

    From a practical standpoint, the proposed JITAIs identified in this study represent general concepts that were perceived favorably by users. Determining the effectiveness of the proposed JITAIs and identifying ideal specific cut-points for each JITAI’s tailoring variables (eg, number of hours or days of no app engagement before delivering an engagement-related JITAI) requires additional investigation. Although preliminary cutoff points for proposed JITAIs can be developed based on qualitative evidence from a usability study such as this, recommended best practices suggest follow-up investigation using complex analytical techniques to refine evidence-based (and potentially adaptive) cutoff points [15]. Implementation of potential JITAIs also depends on feasibility within the digital health platform, as well as confirmation that the final decision rules and cutoff points of the JITAIs do not conflict with one another or with other elements of the intervention’s features or user experience. We plan to refine the details of these JITAIs using a series of microrandomized trials and additional qualitative user feedback [41].

    In addition to leveraging existing evidence-based frameworks, we also leveraged an academic-industry partnership to expedite the incorporation of evidence-based principles into a digital health intervention with potential for rapid dissemination to the general public [63]. Specifically, the commercial app company facilitated participant access to the intervention and iterative mock-ups for the usability testing. The academic researchers led the usability testing itself and applied evidence-based frameworks to guide data collection and analysis. After the identified adaptations are fully built and subsequently tested for effectiveness, they can rapidly be rolled out to several thousand daily users in a much faster time scale than would be possible if an academic group developed a digital health intervention de novo. Our process to identify implementable usability-related adaptations of an existing mental health app can serve as a proof-of-concept method for relatively rapid refinement of other existing digital health interventions in order to better serve specific target populations, especially for apps that already have robust evidence-based content but could benefit from adaptations for specific subpopulations.

    A major strength of this study is the integrated framework-based approach to identify usability-related opportunities for a target user population with unique needs. This approach can serve as a model for other studies, and the usability-related principles that were identified in this study are broadly generalizable for other digital health interventions that are intended for nondigitally native adults with multimorbidity. The robust sample size is also a strength relative to other digital health usability studies [64,65].

    Limitations

    Some limitations are inherent to the study design used. Observer bias (ie, the Hawthorne effect) may have resulted in participants reporting more confidence or demonstrating greater tenacity with technical features than they otherwise would in a real-world setting. Additionally, the iterative nature of our usability testing resulted in some heterogeneity in participant experiences. While participants were not exposed to identical interfaces, this approach is consistent with user-centered and formative usability evaluation methods, in which iterative refinement is prioritized to rapidly identify and address usability barriers [66]. Furthermore, details of some of the usability-related adaptations and JITAIs that were identified in this study are predominantly applicable to the specific mental health app that was tested. Our proposed adaptations also reflect participants’ stated preferences, but quantitative assessment and data-driven refinement was outside the scope of this study. Finally, user preferences among nondigitally native adults may evolve over time as the population’s overall experience with technology continues to grow.

    Given the demographics of our sample, our study findings are likely most generalizable to middle-aged adults in the United States who are relatively digitally proficient and already use the internet routinely throughout the day. As a result, findings from this study may not fully capture the usability needs and other barriers faced by older adults with low digital literacy, which remains an important and underserved segment of the target user population. Interpretation of generalizability is also limited by the self-selected nature of the convenience sample and the lack of a standardized cognitive screening instrument, especially since the prevalence of mild cognitive impairment increases with age.

    Conclusions

    Identifying implementable opportunities to improve usability and prompt meaningful engagement at relevant timepoints holds promise to improve the effectiveness of digital health interventions because intervention attrition is a well-known challenge that is suspected to reduce the effectiveness of digital health interventions on a population level [67-70]. In this study, we used an integrated framework-based approach and user feedback to identify usability-related optimizations to an existing mental health app that are preferred among relatively digitally proficient but nondigitally native adults who have chronic pain and coexisting symptoms of depression or anxiety. Users identified general usability-related adaptations and usability-related JITAIs which were appealing for their potential to (1) provide clarity with regard to how to navigate the digital health intervention, and (2) reduce the user’s burden of independent navigation within the intervention. These usability-related refinement opportunities are generalizable to other digital health interventions intended for older users who are likely to be managing multimorbidity, are not digital natives, and are more likely to have physical or cognitive impairments than younger users (eg, medication reminder apps, daily safety monitoring apps, etc). The integrated framework-based approach taken in this study can also serve as a model for refinement of digital health interventions with other target populations, as well.

    Acknowledgments

    The authors thank Dr Patricia Areán for her contributions to the study conception. Generative artificial intelligence (AI) was not used in any portion of conducting this study or generating this manuscript.

    Funding

    This study was funded by the National Institute of Mental Health (grant R01MH131989; principal investigator ALC). The funder did not play a role in data collection, management, analysis, or interpretation of the data; writing of the report; or the decision to submit the report for publication. Use of Washington University’s instance of REDCap for this study was supported by the Siteman Cancer Center’s National Cancer Institute Cancer Center Support Grant P30CA091842 and by the Washington University Institute of Clinical and Translational Sciences Grant UL1TR002345 from the National Center for Advancing Translational Sciences.

    Data Availability

    The dataset generated and analyzed during this study is available in the National Institute of Mental Health (NIMH) Data Archive (NDA).

    Authors' Contributions

    Conceptualization: ALC, JA

    Methodology: ALC, CYG, SMH, JA

    Study investigation: AM

    Data management: AM

    Analysis: ALC, CYG, AM

    Analysis interpretation: SMH, JA

    Validation: CYG, AM

    Writing: ALC, CYG, AM

    Editing: SMH, JA

    Supervision: ALC, JA

    Funding acquisition: ALC

    Conflicts of Interest

    The academic study team was independent and distinct from the employees, developers, and owners of the Wysa for Chronic Pain intervention. Wysa conceived of and developed the original Wysa for Chronic Pain intervention and provided access to the Wysa for Chronic Pain intervention by the study team and study participants. Based on participant feedback obtained by the study team as part of grant R01MH131989, Wysa created intervention mock-ups to be used in the iterative usability testing of this study. Chaitali Sinha is the chief clinical research and development officer at Wysa, and she is a co-investigator on grant R01MH131989. Wysa was not involved in the development of the study design, conduct of the usability testing, data analysis, final interpretation of the data, or final decision to submit study findings for peer-reviewed publication.

    Multimedia Appendix 1

    Interview guide.

    DOCX File, 28 KB
    1. Gallois CB, Machado MM, de Oliveira CDS, Xavier ACM, Dreher CB, Manfro GG. Is digital psychiatry really for all? A cross-sectional analysis from two randomized clinical trials. Trends Psychiatry Psychother. 2025;47:e20240826. [CrossRef] [Medline]
    2. Schumacher RM. Gran got tech: inclusivity and older adults. J User Exp. 2023;18(2):62-67. URL: https://dl.acm.org/doi/abs/10.5555/3604890.3604891 [Accessed 2026-07-31]
    3. Prensky M. Digital natives, digital immigrants part 1. On Horizon. Sep 2001;9(5):1-6. [CrossRef]
    4. Fortuna KL, Torous J, Depp CA, et al. A future research agenda for digital geriatric mental healthcare. Am J Geriatr Psychiatry. Nov 2019;27(11):1277-1285. [CrossRef] [Medline]
    5. Wang S, Bolling K, Mao W, et al. Technology to support aging in place: older adults’ perspectives. Healthcare (Basel). 7(2):60. [CrossRef] [Medline]
    6. Perrin A. Mobile technology and home broadband 2021. Pew Research Center; 2021. URL: https://www.pewresearch.org/internet/2021/06/03/mobile-technology-and-home-broadband-2021/ [Accessed 2026-07-24]
    7. Wildenbos GA, Peute L, Jaspers M. Aging barriers influencing mobile health usability for older adults: a literature based framework (MOLD-US). Int J Med Inform. Jun 2018;114:66-75. [CrossRef] [Medline]
    8. Wagner N, Hassanein K, Head M. The impact of age on website usability. Comput Hum Behav. Aug 2014;37(Aug):270-282. [CrossRef]
    9. Romano Bergstrom JC, Olmsted-Hawala EL, Jans ME. Age-related differences in eye tracking and usability performance: website usability for older adults. Int J Hum Comput Interact. Aug 3, 2013;29(8):541-548. [CrossRef]
    10. Americans check their phones 96 times a day. Asurion URL: https://www.asurion.com/press-releases/americans-check-their-phones-96-times-a-day/ [Accessed 2026-07-24]
    11. Bartels SJ, DiMilia PR, Fortuna KL, Naslund JA. Integrated care for older adults with serious mental illness and medical comorbidity: evidence-based models and future research directions. Psychiatr Clin North Am. Mar 2018;41(1):153-164. [CrossRef] [Medline]
    12. Cheng AL, Gou CY, Martin A, Hartz SN, Abraham J. Design preferences for mental health apps among nondigitally native adults with chronic pain: qualitative analysis. JMIR Mhealth Uhealth (forthcoming). 2026. [CrossRef]
    13. Nahum-Shani I, Smith SN, Spring BJ, et al. Just-in-Time Adaptive Interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med. May 18, 2018;52(6):446-462. [CrossRef] [Medline]
    14. Teepe GW, Da Fonseca A, Kleim B, et al. Just-in-time adaptive mechanisms of popular mobile apps for individuals with depression: systematic app search and literature review. J Med Internet Res. Sep 28, 2021;23(9):e29412. [CrossRef] [Medline]
    15. van Genugten CR, Thong MSY, van Ballegooijen W, et al. Beyond the current state of just-in-time adaptive interventions in mental health: a qualitative systematic review. Front Digit Health. 2025;7(1460167):1460167. [CrossRef] [Medline]
    16. DeVeaugh-Geiss AM, West SL, Miller WC, Sleath B, Gaynes BN, Kroenke K. The adverse effects of comorbid pain on depression outcomes in primary care patients: results from the ARTIST trial. Pain Med. May 2010;11(5):732-741. [CrossRef] [Medline]
    17. Roughan WH, Campos AI, García-Marín LM, et al. Comorbid chronic pain and depression: shared risk factors and differential antidepressant effectiveness. Front Psychiatry. 2021;12:643609. [CrossRef] [Medline]
    18. Gerrits MMJG, van Oppen P, Leone SS, van Marwijk HWJ, van der Horst HE, Penninx BW. Pain, not chronic disease, is associated with the recurrence of depressive and anxiety disorders. BMC Psychiatry. Jun 25, 2014;14:187. [CrossRef] [Medline]
    19. Gerrits MMJG, Vogelzangs N, van Oppen P, van Marwijk HWJ, van der Horst H, Penninx BWJH. Impact of pain on the course of depressive and anxiety disorders. Pain. Feb 2012;153(2):429-436. [CrossRef] [Medline]
    20. Dahlhamer J, Lucas J, Zelaya C, et al. Prevalence of chronic pain and high-impact chronic pain among adults - United States, 2016. MMWR Morb Mortal Wkly Rep. Sep 14, 2018;67(36):1001-1006. [CrossRef] [Medline]
    21. Bhattarai P, Phillips JL. The role of digital health technologies in management of pain in older people: an integrative review. Arch Gerontol Geriatr. 2017;68:14-24. [CrossRef] [Medline]
    22. Carlo AD, Hosseini Ghomi R, Renn BN, Strong MA, Areán PA. Assessment of real-world use of behavioral health mobile applications by a novel stickiness metric. JAMA Netw Open. Aug 3, 2020;3(8):e2011978. [CrossRef] [Medline]
    23. Carlo AD, Hosseini Ghomi R, Renn BN, Areán PA. By the numbers: ratings and utilization of behavioral health mobile applications. NPJ Digit Med. 2019;2(54):54. [CrossRef] [Medline]
    24. Torous J, Bucci S, Bell IH, et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry. Oct 2021;20(3):318-335. [CrossRef] [Medline]
    25. Torous J, Linardon J, Goldberg SB, et al. The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality. World Psychiatry. Jun 2025;24(2):156-174. [CrossRef] [Medline]
    26. Abd-Alrazaq AA, Alajlani M, Ali N, Denecke K, Bewick BM, Househ M. Perceptions and opinions of patients about mental health chatbots: scoping review. J Med Internet Res. Jan 13, 2021;23(1):e17828. [CrossRef] [Medline]
    27. Mohr DC, Kwasny MJ, Meyerhoff J, Graham AK, Lattie EG. The effect of depression and anxiety symptom severity on clinical outcomes and app use in digital mental health treatments: meta-regression of three trials. Behav Res Ther. Dec 2021;147:103972. [CrossRef] [Medline]
    28. Arean PA, Hallgren KA, Jordan JT, et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res. Dec 20, 2016;18(12):e330. [CrossRef] [Medline]
    29. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. Jan 2018;47(1):1-18. [CrossRef] [Medline]
    30. Rathbone AL, Prescott J. The use of mobile apps and SMS messaging as physical and mental health interventions: systematic review. J Med Internet Res. Aug 24, 2017;19(8):e295. [CrossRef] [Medline]
    31. Mojtabai R, Olfson M, Sampson NA, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. Aug 2011;41(8):1751-1761. [CrossRef] [Medline]
    32. Schure M, McCrory B, Tuchscherer Franklin K, Greist J, Weissman RS. Twelve-month follow-up to a fully automated internet-based cognitive behavior therapy intervention for rural adults with depression symptoms: single-arm longitudinal study. J Med Internet Res. Oct 2, 2020;22(10):e21336. [CrossRef] [Medline]
    33. Gureje O, Von Korff M, Simon GE, Gater R. Persistent pain and well-being: a World Health Organization study in primary care. JAMA. Jul 8, 1998;280(2):147-151. [CrossRef] [Medline]
    34. Ruoff GE. Depression in the patient with chronic pain. J Fam Pract. Dec 1996;43(6 Suppl):S25-S33. [Medline]
    35. Zelaya CE, Dahlhamer JM, Lucas JW, Connor EM. Chronic pain and high-impact chronic pain among U.S. adults, 2019. NCHS Data Brief. Nov 2020;(390):1-8. [Medline]
    36. Sharpe L, McDonald S, Correia H, et al. Pain severity predicts depressive symptoms over and above individual illnesses and multimorbidity in older adults. BMC Psychiatry. May 4, 2017;17(1):166. [CrossRef] [Medline]
    37. Sheng J, Liu S, Wang Y, Cui R, Zhang X. The link between depression and chronic pain: neural mechanisms in the brain. Neural Plast. 2017;2017:9724371. [CrossRef] [Medline]
    38. Kamper SJ, Apeldoorn AT, Chiarotto A, et al. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: cochrane systematic review and meta-analysis. BMJ. Feb 18, 2015;350:h444. [CrossRef] [Medline]
    39. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. Sep 2001;16(9):606-613. [CrossRef] [Medline]
    40. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. May 22, 2006;166(10):1092-1097. [CrossRef] [Medline]
    41. Cheng AL, Abraham J, Hartz SM, Laber EB, Miller JP. Evaluating and optimizing just-in-time adaptive interventions in a digital mental health intervention (Wysa for Chronic Pain) for middle-aged and older adults with chronic pain: protocol for a series of randomized trials. JMIR Res Protoc. Sep 17, 2025;14:e77532. [CrossRef] [Medline]
    42. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. Jul 2019;95:103208. [CrossRef] [Medline]
    43. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. Apr 2009;42(2):377-381. [CrossRef] [Medline]
    44. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [CrossRef]
    45. Cheng AL, Agarwal M, Armbrecht MA, Abraham J, Calfee RP, Goss CW. Behavioral mechanisms that mediate mental and physical health improvements in people with chronic pain who receive a digital health intervention: prospective cohort pilot study. JMIR Form Res. Nov 17, 2023;7:e51422. [CrossRef] [Medline]
    46. Leo AJ, Schuelke MJ, Hunt DM, et al. A digital mental health intervention in an orthopedic setting for patients with symptoms of depression and/or anxiety: feasibility prospective cohort study. JMIR Form Res. Feb 21, 2022;6(2):e34889. [CrossRef] [Medline]
    47. Leo AJ, Schuelke MJ, Hunt DM, Miller JP, Areán PA, Cheng AL. Digital mental health intervention plus usual care compared with usual care only and usual care plus in-person psychological counseling for orthopedic patients with symptoms of depression or anxiety: cohort study. JMIR Form Res. May 4, 2022;6(5):e36203. [CrossRef] [Medline]
    48. Sinha C, Cheng AL, Kadaba M. Adherence and engagement with a cognitive behavioral therapy-based conversational agent (Wysa for Chronic Pain) among adults with chronic pain: survival analysis. JMIR Form Res. May 23, 2022;6(5):e37302. [CrossRef] [Medline]
    49. Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. Apr 7, 2021;16(1):36. [CrossRef] [Medline]
    50. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. Jun 6, 2019;14(1):58. [CrossRef] [Medline]
    51. Kroenke K, Spitzer RL, Williams JBW, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics. 2009;50(6):613-621. [CrossRef] [Medline]
    52. Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: a pragmatic framework. Health Psychol. Dec 2015;34S:1209-1219. [CrossRef] [Medline]
    53. Mohr DC, Schueller SM, Montague E, Burns MN, Rashidi P. The behavioral intervention technology model: an integrated conceptual and technological framework for eHealth and mHealth interventions. J Med Internet Res. Jun 5, 2014;16(6):e146. [CrossRef] [Medline]
    54. Zingg A, Singh T, Myneni S. Towards digestible digital health solutions: application of a health literacy inclusive development framework for peripartum depression management. AMIA Annu Symp Proc. 2021;2021:1274-1283. [Medline]
    55. Harrison Ginsberg K, Babbott K, Serlachius A. Exploring participants’ experiences of digital health interventions with qualitative methods: guidance for researchers. J Med Internet Res. Nov 28, 2024;26:e62761. [CrossRef] [Medline]
    56. Colbenson GA, Cook DA, Stephenson CR. Learner engagement, teaching effectiveness, and digital proficiency in in-person versus livestream continuing medical education. J Contin Educ Health Prof. 2026;46(2):88-94. [CrossRef] [Medline]
    57. Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singap. Mar 1994;23(2):129-138. [Medline]
    58. Fouyaxis J, Bidargaddi N, Du W, Looi JCL, Lipschitz J. Critical design decisions and user demographics in enhancing real-time digital mental health interventions: a systematic review. Digit Health. 2024;10:20552076241306782. [CrossRef] [Medline]
    59. Rising CJ, Jensen RE, Moser RP, Oh A. Characterizing the US population by patterns of mobile health use for health and behavioral tracking: analysis of the National Cancer Institute’s Health Information National Trends Survey data. J Med Internet Res. May 14, 2020;22(5):e16299. [CrossRef] [Medline]
    60. Jaana M, Paré G. Comparison of mobile health technology use for self-tracking between older adults and the general adult population in Canada: cross-sectional survey. JMIR Mhealth Uhealth. Nov 27, 2020;8(11):e24718. [CrossRef] [Medline]
    61. Paré G, Leaver C, Bourget C. Diffusion of the digital health self-tracking movement in Canada: results of a national survey. J Med Internet Res. May 2, 2018;20(5):e177. [CrossRef] [Medline]
    62. Strain T, Wijndaele K, Brage S. Physical activity surveillance through smartphone apps and wearable trackers: examining the UK potential for nationally representative sampling. JMIR Mhealth Uhealth. Jan 29, 2019;7(1):e11898. [CrossRef] [Medline]
    63. Hall SS, Hastings O, Prentice KM, et al. Principles of industry-academic partnerships informed by digital mental health collaboration: mixed methods study. JMIR Ment Health. Sep 10, 2025;12:e77439. [CrossRef] [Medline]
    64. Sin J, Woodham LA, Henderson C, Williams E, Sesé Hernández A, Gillard S. Usability evaluation of an eHealth intervention for family carers of individuals affected by psychosis: a mixed-method study. Digit Health. 2019;5:2055207619871148. [CrossRef] [Medline]
    65. Stonbraker S, Cho H, Hermosi G, Pichon A, Schnall R. Usability testing of a mHealth app to support self-management of HIV-associated non-AIDS related symptoms. Stud Health Technol Inform. 2018;250:106-110. [Medline]
    66. Marcilly R, Monkman H, Pelayo S, Lesselroth BJ. Usability evaluation ecological validity: is more always better? Healthcare (Basel). Jul 16, 2024;12(14):1417. [CrossRef] [Medline]
    67. Eysenbach G. The law of attrition. J Med Internet Res. Mar 31, 2005;7(1):e11. [CrossRef] [Medline]
    68. Owen JE, Jaworski BK, Kuhn E, Makin-Byrd KN, Ramsey KM, Hoffman JE. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health. 2015;2(1):e7. [CrossRef] [Medline]
    69. Laing BY, Mangione CM, Tseng CH, et al. Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: a randomized, controlled trial. Ann Intern Med. Nov 18, 2014;161(10 Suppl):S5-12. [CrossRef] [Medline]
    70. Y Wang A, Vereschagin M, G Richardson C, et al. Examining the effects of engagement with an app-based mental health intervention: a secondary analysis of a randomized control trial with treatment non-compliance. Int J Ment Health Syst. Oct 9, 2025;19(1):30. [CrossRef] [Medline]

    ‎
    BIT: behavioral integration technology
    FRAME-IS: Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies
    JITAI: just-in-time adaptive intervention
    REDCap: Research Electronic Database Capture

    Edited by Stephanie Law; submitted 07.Nov.2025; peer-reviewed by Francesca Bruni, Silvana Schwerz Funghetto; final revised version received 15.Jul.2026; accepted 15.Jul.2026; published 11.Aug.2026.

    Copyright

    © Abby L Cheng, Christine Y Gou, Adriana Martin, Sarah M Hartz, Joanna Abraham. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 11.Aug.2026.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.